Sample Size Considerations for Multiple Comparison Procedures in ANOVA

Adequate sample sizes for omnibus ANOVA tests do not necessarily provide sufficient statistical power for post hoc multiple comparisons typically performed following a significant omnibus F test. Results reported support a comparison-of-most-interest approach for sample size determination in ANOVA based on effect sizes for multiple comparisons. Introduction The determination of an appropriate sample size is an often difficult, but critically important, element in the research design process. One of the chief functions of experimental design is to ensure that a study has adequate statistical power to detect meaningful differences, if indeed they exist (e.g., Hopkins & Hopkins, 1979). There is a very good reason why researchers should worry about statistical power a priori: If researchers are going to invest time and money in carrying out a study, then they would want to have a reasonable chance, perhaps 70% or 80%, to find a statistically significant difference between groups if it does exist in the population. Thus, a priori power, the probability of rejecting a null hypothesis that is indeed false, will inform researchers about how many subjects per group will be needed for adequate power (Light, Singer & Willett, 1990). Among the most important matters impacting the choice of sample size is the